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FINAL  REPORT  6/1/S6  -  7/31/89  AFOSR- 86- 0153 
P.I.  Sheldon  M.  Ross,  University  of  California,  Berkeley 

During  this  period  10  papers  supported  by  the  grant  were  published.  These 
papers  fell  into  4  general  categories: 

1;  Approximations  in  Stochastic  Reliability  Systems 
2:  Software  Reliability 

3:  Variance  Reduction  Techniques  in  Simulation 
4;  General  Stochastic  Models. 

Ve  now  outline  these  areas  and  the  results  obtained. 

1.  Approximations  in  Stochastic  Relaibilitv  Systems 

One  is  often  inueiesred  in  determining  E[X(t)],  the  expected  state  at 
time  t  of  some  stochastic  system.  In  grant  supported  research,  we  have 
developed  an  approach  to  approximating  this  by  the  value  E[X(Y^+. .  .+Y^^)] 
whe^'e  the  Y^,  i=l,...,n  are  independent  and  identically  distributed 
exponential  random  variables  with  mean  t/n  which  are  also  independent  of  the 
underlying  process  of  interest.  In  the  paper  [l]  this  approach  was  utilized  to 
approximate  the  renewal  function  m(t),  equal  to  the  expected  number  of 
renewals  by  time  t.  It  was  shown  in  [1]  that  if  we  set 

e.  =  E[X^e'"^/^] 

where  X  represents  an  interarrival  time  of  the  process,  then  m(t)  can  be 
approximated  by  the  value  m^^  which  is  obtained  from  the  recursion 

"i  "  VC  -  ‘'o' 

and ,  for  r  =  2, . . .  ,n 


r- 1 

It  was  shown  in  [1]  that 

n.^  ^  E[N(Y^  .....  YJ] 

where  Yp  . . .  ,Yj^  are  independent  e.xponentials  with  mean  t/n  which  are 
indepedndent  of  the  renewal  process.  In  addition,  it  was  shown  in  [l]  that  if 
the  renewal  function  is  continuous  at  t  then  the  approximation  converges  to 
m(t)  as  n  approaches  infinity.  Also,  if  the  intorarrival  distribution  is 
Decreasing  Failure  Rate  then  the  sequence  of  approximations  m^  constitutes  an 
increasing  sequence  of  lower  bounds.  Other  renewal  theoretic  quantities  that 
were  also  approximated  in  [l]  were  the  integrated  renewal  function,  the  mean 
and  distribution  of  the  age  and  excess  of  the  renewal  process  at  t,  and  the 
probability  mass  funciion  ox  the  number  of  renewals  by  time  t. 

The  approach  of  approximating  E[X(t)]  by  E[X(Yj.. .  ..Y^^)]  ,  whcie  ciie 
Y-  are  independent  exponentials  with  mean  t/n,  was  continued  in  [2]  which 
considered  continuous  time  Markov  chains.  In  the  case  where  the  quantity  of 
interest  was  the  transition  probability  F^.(t)  it  was  shown  that  the 
approach  resulted  in  the  approximation  being  given  by  the  i-j  element  in  the 
matrix  (I  -  R/A)  ^  where  R  is  the  matrix  of  instantaneous  transition  rates 
(with  the  i^^^  diagonal  element  being  equal  to  the  negative  the  rate  at 
which  the  chain  departs  from  state  i),  I  is  the  identity  matrix,  and  A  =n/t. 
It  was  shown  in  [2]  that  these  appx'oximations  converge  to  the  exact  values  as 
n  approaches  infinity. 


In  addition,  approximations  to  the  mean  time  that  the  process  spends  in  a  in  Foi^ 
given  state  by  time  t,  called  the  mean  occupation  time,  and  the  mean  number  of 
transitions  to  that  state  by  time  t  were  also  presented  in  [2].  The  mean 
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occupation  time  is  quite  important  in  applications.  For  instance,  consider  the 
queueing  system  having  k  e.xponential  servers  in  which  customers  arrive 
according  to  a  Poisson  process  but  only  eneter  the  system  if  the  number  of 
waiting  customers  is  less  than  some  given  constant  m.  A  quantity  of  interest 
is  the  e.xpected  number  of  lost  customers  by  time  t  which  is  easily  shown  to  be 
equal  to  the  arrival  rate  multiplied  by  the  expected  time  at  which  there  are  m 
waiting  customers  by  t. 

The  follow-up  note  [3]  showed  how  to  improve  the  efficiency  of  the 
approximations  of  the  mean  occupation  times  given  in  [2],  by  utilizing  a 
"doubling- up"  approach.  This  approach  only  requires  computation  of  P*^,  P"^, 

2k 

...  ,  p-  for  a  given  matrix  P  rather  than  all  of  the  powers  P^  , 
i=2,...,2^  as  would  have  been  needed  using  the  initial  approach  presented  in 
[2]- 

2.  Software  Relaibilitv 

Consider  a  situation  in  which  there  are  an  unknown  number  of  errors 
contained  in  a  piece  of  software.  Suppose  that  each  error,  independently, 
causes  mistakes  according  to  a  Poisson  process  whose  rate  depends  on  the 
error.  The  problem  of  interest  is  to  observe  the  system  for  a  fixed  time, 
noting  all  mistakes  and  assigning  these  mistakes  to  the  errors  that  caused 
them,  and  then  use  the  resulting  data  to  estimate  -  °oisson  rates  of  the 
errors  discovered.  (The  importance  of  this  problem  :  at  it  helps  us  gauge 

tlie  quality  of  the  original  software).  In  other  words,  the  resultant  data  will 
be  that  k  different  errors  are  observed  to  have  caused  N^,  ...  ,  N^.  mistakes 
respectively.  Whereas  the  maximum  likelihood  estimate,  which  estimates  the 


mistake  rate  corresponding  to  the  error  that  caused  mistakes  by  time  t  by 
the  value  N^/t  ,  i=l,...,k,  is  a  natui'al  estimator  to  consider  it  was  shown 
by  Lieberman  and  Ross  in  [4]  that  substantial  improvements  can  be  made. 
Specifically,  it  was  shown  that  by  estimating  (by  a  method  presented  by  Ross 
in  the  earlier  paper  [5])  the  sum  of  the  mistake  rates  of  all  errors  that  did 
not  cause  any  mistakes  by  time  t  and  then  using  this  value  to  reduce  the 
maximum  likelihood  estimates  we  can  improve  upon  the  maximum  likelihood 
estimates.  In  addition,  it  was  shown  that  by  utilizing  an  approach  that 
estimates  a  vector  of  means  not  by  their  maximum  likelihood  estimates  but  by 
an  approach  that  "weights  these  maximum  likelihood  estimates  towards  their 
average  value"  also  results  in  an  improved  estimator.  In  addition,  it  was 
shown  in  [4] ,  by  a  simulation  study,  that  doing  both  of  the  above 
simultaneously  results  in  the  greatest  improvement. 

In  the  paper  [6],  Ross  along  with  his  colleagues  Dcrman  and  Lieberman, 
considered  a  situation  in  which  a  large  lot  of  items  is  to  be  sampled  and 
inspected  for  Lhe  purpose  of  ascertaining  its  number  of  defective  units.  They 
supposed  a  total  of  k  inspectors,  with  each  inspector  having  a  different 
unknown  probability  of  detecting  a  defective  item  that  he  inspects.  They 
supposed  that  the  sample  of  items  to  be  inspected  is  to  be  broken  intr> 
mutually  exclusive  subsamples,  with  each  of  the  k  inspectors  being  assigned  to 
inspect  a  number  of  these  subsamples;  and  then  presented  a  way  -  based  on  the 
method  of  moments  -  to  estimate  the  total  number  of  defective  units.  In 
addition,  the  design  problem  of  deciding  tlie  amount  of  sampling  overlap  was 
also  considered. 


3.  Variance  Reduction  in  Simulation 

Consider  a  queueing  system  in  which  customers  enter  service  according  to 
their  order  of  arrival  and  in  which  the  arrival  process  is  independent  of  the 
sequence  of  service  times.  Suppose  one  wants  to  use  simulation  to  estimate  the 
expected  sum  of  the  times  that  the  first  n  customers  spend  in  such  a  system. 

If  we  let 

=  the  time  in  the  system  of  customer  i 

=  history  of  the  process  up  to  the  moment  that  customer  i  arrives 

n 

then  the  raw  simulation  estimator  from  a  single  run  would  be  S  D-  ;  however. 

i=l  ^ 

it  was  shown  in  the  grant  supported  research  [7]  that  a  better  estimator,  in 

n 

the  sense  of  having  a  smaller  mean  square  error,  is  E  E[D-(I1.]  .  For 

i=l  ^  ^ 

instance,  in  the  case  of  a  k  server  system  in  which  the  service  time  is 
exponential  with  mean  fi, 

EfDJIlj]  =  ^  .  (,Nj  +  1  -  k)Vk 

where  is  the  number  of  customers  in  the  system  at  the  moment  customer  i 

arrives;  and  using  the  sum  of  the  first  n  terms  of  this  typr  results  in  a 

n  n 

better  estimate  of  E[  E  DJ  than  does  the  raw  simulation  estimator  ED-.  In 

i=l  ^  i=l  ^ 

the  case  where  there  is  a  single  server  having  an  arbitrary  service 
distribution  we  have  that 

E[DJII.]  =  N./i  +  //(a.) 

where  a^  is  the  amount  of  time  the  customer,  in  service  when  i  arrives,  has 
already  spent  in  service  when  i  arrives,  and  //(a)  is  the  expected  remaining 
service  time  of  a  customer  that  has  already  spent  a  time  units  in  service. 

In  [8]  Ross  considered  the  problem  of  using  the  output  of  a  simulation  to 


estimate  the  mean  number  of  renewals  by  some  fixed  time  t.  That  is,  suppose 
one  continually  generates  a  sequence  of  independent  non- negative  random 
variables  having  the  distribution  F  -  representing  the  interarrival  times  of  a 
counting  process  -  until  their  sum  exceeds  t;  and  suppose  we  want  to  utilize 
this  to  estimate  the  expected  number  of  events  by  time  t.  Thus,  any  simulation 
run  will  generate  the  value  of  N(t)  +  1  of  these  interevent  times  -  where 
N(t)  represents  the  number  of  events  that  occur  by  time  t. 

By  niahing  use  nf  thp  identity 

N(t)+1 

E[  t  (X.  -  p)]  =  0, 

i-1  ^ 

where  is  the  i^^  interevent  time,  and  /i  =  E[X.],  the  controlled 
estimator 

N(t)+1 

N(t)  +  c[  (X.  -  /.)]  =  N(t)  +  C[t  +  Y(t)  -  p.N(t)  -  p] 

where  Y(t)  ,  called  the  excess  at  t,  represents  the  time  from  t  until  the 
next  event,  was  considered  in  [8].  The  value  of  c  leading  to  the  smallest 
variance  can  be  estimated  by  the  simulation.  In  addition,  it  was  shown  that, 
for  t  large,  the  best  estimator  of  this  type  chose  c  approximately  equal  to 
l//i. 

It  was  then  shown  in  [8]  that,  for  any  value  of  c,  replacing  Y(t)  in 
the  above  by  E[Y(t)lA(t)]  results  in  a  reduction  of  variance,  where  A(t), 
referred  to  as  the  age  of  the  renewal  process  at  t,  is  the  time  at  t  since  the 
last  event  prior  to  t. 


4.  General  Stochastic  Models 

In  [9]  Ross  critiqued  an  influential  paper  of  Raup  and  Sepkoski 
("Periodicity  of  Extinctions  in  the  Geologic  Past,"  Proceedings  of  the 
National  Academy  of  Sciences,  81,  801-801,  1984).  The  Raup- Sepkoski  paper 
analyzed  data  relating  the  proportion  of  existing  families  that  became  extinct 
in  39  time  periods  (of  average  length  6.2  million  years).  They  claimed  that 
this  data  indicated  a  periodicity  of  mass  extinctions  and  thus  invalidated  the 
previous  held  belief  that  such  data  behaved  as  a  random  walk  whose  incremental 
change  distribution  is  symmetric  about  0.  However,  it  was  noted  in  [9]  that 
the  statistical  analysis  presented  in  the  Raup- Sepkoski  was  flawed  in  that  the 
test  it  utilized  is  not  meaningful  when  the  alternative  hypothesis  is  the 
random  walk  model.  It  was  then  shown  in  [9],  by  a  nonparametric  analysis,  that 
the  random  walk  is  perfectly  consistent  with  the  data. 

The  Raup- Sepkoski  paper  defined  an  extinction  peak  to  occur  whenever  the 
number  of  extinctions  in  a  period  exceeded  that  of  its  two  immediate 
neighboring  periods.  This  led  Ross,  in  [10],  to  a  study  of  the  following 
problem.  Let  X^,...  be  a  sequence  of  random  variables  and  say  that  a  peak 
occurs  at  time  n  if  X^  ^  <  X^^  <  X^^^  .  When  the  random  sequence  constitutes  a 
random  walk  whose  incremental  change  distribution  is  symmetric  about  0  then, 
as  was  noted  in  [9] ,  the  process  of  peaks  constitutes  a  renewal  process. 
However,  when  the  X^^  constitute  a  random  sample  from  a  continuous  distribution 
this  is  no  longer  true.  Indeed,  in  this  situation  the  times  between  successive 
peaks  are  neither  independent  not  identically  distributed.  The  process  of 
peaks,  in  this  latter  case,  was  analyzed  in  [10].  It  was  shown  that  N(n),  the 


number  of  peaks  by  time  n,  is  asymptotically  normally  distributed  with  mean 
(n-l)/3  and  variance  (2n+4)/45  .  In  addition,  it  was  shown  that,  with 
probability  1,  limit  N(n)/n  =  i/3  .  Finally,  it  was  argued  in  [10]  that  the 

n-too 

proportion  of  interpeak  times  that  equal  j  converges  to  a  constant  vnluc  which 
was  then  evaluated  for  a  variety  of  j. 

In  [11]  Ross  considered  the  classical  communications  problem  in  which  the 
numbers  of  messages  that  arrived  in  each  distinct  time  period  were 
independently  and  identically  distributed.  It  was  supposed  that  each  arriving 
message  will  transmit  at  the  end  of  the  period  in  which  it  arrives.  If  e.^actly 
1  message  is  transmitted  then  the  transmission  is  successful  and  the  message 
leaves  the  system.  However,  if  at  any  time  2  or  more  messages  simultaneously 
transmit  then  a  collision  is  deemed  to  occur  and  these  messages  remain  in  the 
system.  Once  a  message  is  involved  in  a  collision  it  will,  independently  of 
all  else,  transmit  at  the  end  of  each  additional  period  with  probability  p  - 
the  so-called  Aloha  protocol.  In  [il]  an  extremely  elementary  argument  was 
used  to  show  that  such  a  system  is  asymptotically  unstable  in  that  the  number 
of  successful  transmissions  is  finite  with  probability  1.  The  same  argument 
was  also  used  to  show  that  this  result  is  also  true  for  those  back- off 
protocols  whose  transmission  probabilities  are  bounded  away  from  0. 
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